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ABSTRACT 

This study examined a state-of-the-art computer-based 
integral ^d learning system to determine its conformance to research 
findin'^s on effective learning practices; to describe its 
a!"Dier. entation in an elementary school setting; and to assess changes 
in students' achievement in either mathematics or reading. 
Pretest/posttest data were .supplemented by observations, interviews, 
document reviews, and teacher and student questionnaires. Results of 
the achievement tests showed that students' test scores improved; 
however, treatment differences in reading were significant only ir. 
th^ white group for comprehension. Significant differences in the 
mathematics component were restricted co race and race by sex. 
Overall, this ctudy supported research findings that indicate 
computer programs whose management is not an additional burden on the 
classroom teacher, match the local curricula, and address learning 
theory research can be implemented successfully. (23 references) 
(Author) 
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This study examined a state-of-the-art computer-based integrated 
learning system to determine* its conformance to research findings on effective 
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learning practices, to describe its implementation in an elementary setting, 
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paa^ and to assess changes in students' achievement in either mathematics or 

reading. Pretest/poottest data were supplemented by observations, interviews, 
document reviews, and teacher and student questionnaires. Results of tne 
^5 achievement tests showed that students' test scores improved; however, 



treatment differences in reading were significant only in the white group for 
comprehension. Significant dif f .-rences in the mathematics component were 
restricted ro race and race by sex. Overall, this study supported research 
findings that indicate comp^ ter programs whose management is not an additional 
burden on the cl«issroom teacher, match the local curricula, and address 
le arning theory research can be implemented successfully. (Keywords: computer- 
assisted inscruction, integrated learning systems, achievement, evaluation) 
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One increasingly prevalent use of computers in education has been in the 
area of computer-assisted instruction (CAI). In an ideal mode, specially 
designed computer prograirs introduce the student tc a concept or skill, 
provide interactive practice, and reinforce learning >ith appropriate 
feedback. As technology has improved, these CAI programs have become mor^ 
sophisticated ir terms of the speed, graphics, branching capabilities, and 
augmented sound and animation. Other improvements include the incorporation 
of principles of learning theory in the design of software and a correlation 
of accivicies to the developmental stages of children's cognition. Most 
recently, microcomputers or terminals have been networked to a high-speed 
central system or "host" which manages an interactive program of lessons, 
tests, and record-keeping. Tbese "integrated learning systems" have addressed 
many of the earlier problen,^ associated with CAI use and liave incorporated 
much of what is considered sound learning theory into their design. 

GOAL OF THE STUDY 

Much vork has been conducted on various aspects of learning theory and 
their relationship to achievement (Skinner, 1968; Piaget, 1966; Bigge, 1982; 
Thorndike, 1906). These basic tenets of learning theory, such as notivation, 
active participation, and use of feedback, are currently being related to the 
design end implementation of many computer-based instructional programs 
(Baker, 1978; Chandler, 1984, Dillon & Steinberg, 1986; Poison & Richardson, 
1988). The premise of such work is that if compurer-based instructional units 
are designed and implemented according to established theories of learning, 
increased achievement will result. This study was designed to investigate 
that premise by exploring those aspects of learning theory that, v^hen 
incorporated incr^ a computerized integrated learning system, can contribute to 
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a successful learning experience for a student. One such system was analyzed 
for its conformance to these practices. Achievement results of students using 
the system in either reading or mathematics were measured and compared to 
those of a control group. 

PROGRAM DESCRIPTION 
A Computer Assisted Instruction (CAI) system developed by Education 
Systems Corporation (now Jos tens Learning Corporation) in San Diego, 
California, was pilot tested during 1987-88 at Denbar Elementary, a K, 4-6 
grade schcol located in a large school system near a metropolitan area. The 
Education System Corporation (ESC) Integrated Learning System consists of 
mathematics and reading curricula for grades 1-6, a management system, and a 
testing system. The CSC Curricula parallel the scope and sequence of ma^ r 
textbooks, including those used in thip particular school system, and provide 
continuous niulti- sensory instruction through self -paced lessons enhanced with 
color, voice, music, friendly characters, and graphics. The system at Denbar 
is set up as a lab of 28 Tandy i.000 computer stations networked to ^ Tandy 
3000 host. The curricula and management system are contained on one CD 
(compact disk) which is updated by ESC on a regular basis. One classroom at 
Denbar was set up as the ESC lab; an operator trained by ESC managed the 
system. Students were scheduled by class for three 20-minute sessions weekly 
from October to May. The teacher accompanied the suudcnts and was expected to 
monitor and assist students as necessary. When students entered the lab, they 
went directly to their assigned computer stations already loaded with the 
individualized, self -paced lessons. These lessons included tutorials, 
practice exercises, and periodic tasting. As students completed each lesson, 
the next activity was accessed automatically. Use of the system was a 
complement to the regular curriculum program. 



SIGNIFICANCE OF THE STUDY 

Henry J. Be:ker of Johns Hopkins University discusses the need for more 
current, well-concucted research studies in a report in which he reviews bot-h 
outdated and recent research and ^^hen urges researchers to address the 
studies' weaknesses as they design new projects (1988). "To be most helpful, 
studies need to describe in detail how computers (and control treatments) were 
implemented at each site, and to do so in some kind of standardized way..." 
(p. 23). Bozeman and House (1988) reiterate this need for descriptive 
research on the implementation of a program, saying that although we need be 
concerned about effectiveness, it is "essential to first know how, and the 
extent to which, the program was actually implemented" (p. 85). This need for 
"full-disclosure" is clarified by Nira Hativa (1988) when she stresses the 
need for descriptions of the major features of CAI systems including hardware, 
software, management 3ysteru^, and method of daily operation to increap.e our 
knowledge of the interaction of such system characteristics with students. 
Gary Bitter (1986) also urges researchers to consider "software and its 
capabilities, application and their abilities to facilitate learning, and the 
many features of the microcomputer and their effect on children" (p. 33). 

This study is significant for three reasons. First, i^ is based on the 
use of an integraced learning system, a concept recognized as the direction 
many school systems are moving in using computers for instruction. Second, 
although the study was conducted in a school environment, its design 
encorporates conditions that meet many of the criteria fo'^ effective and 
worthwhile CAI research (e.g., randon assignment of students to classes and 
classes to treatment; a year-long study to reduce the novelty effect; 
standardized measuring instruments; and the elimination of che novelty effect 



with a design in which both groups used the computeL). Third, rather than 
^.ddrtssing a single component of CAI design or one factor in achievement, the 
s-udy focuses on the overall use of a CAT. system within a particular context. 

METHODS AND PROCEDURES 
Sample and Population Description 
Denbar Elementary is a kindergarten, 4th-6th grade elementary school 
located on an army base and operated by the local school system. The school 
has a population of approximately 460, of which 37Z are minority. Prior to 
the 1987-88 school year, students had used Atari com' alters in the cla.^-sroom 
for Logo, word processing, and independent CAI programs. 

Research Design 

This study employed a pretest/posttest control group design. Fourth 
through sixth grade students at Denbar were grouped through a stratified 
random assignment procedure used to ensure an ^qual distribution by sex, 
minority, and ability levels. In addition to the random student assignments, 
two of the classes at each grade Isvel were randomly selecueu co use the 
reading curriculum and the two others to use the mathematl'zs curiiculura. 
Because of the above procedure, the two classes usi g the riathema^:ics 
curriculum were the control group for the reading curriculum ond vice versa. 

Data Collection 

Data were collected through various means. Fieldnotes were taken by the 
researcher at each of her morthly visits and the software was reviewed 
formally and ir^fornpally. ESC documentation, including the correlation of 
objectives to the school's program, lab forms on special requests and problems 
encountered, and student histories were reviewed. All teachers were 
interviewed, and both t^.achers and students completed Likert scale 



questionnaires. Test data from both the BSI (a computerized adaptive test 
developed by ESC) and from McGraw Hill's Comprehensive Test of Basic Skills 
(CTBS) were analyzed pre and post. 

RESULTS 

Document Rev iew: Of the 205 mathematics objectives itemized by the 
school system framework, ESC matched 182 or 89%. Ninety percent were 
presented at the numerical level associated with the grade This high match 
meant that teachers were able to view the lab as a supplem. at rather than 
addition to what they were responsible for teaching in the classroom. 

The review of the lab documents showed that although teachers 
increasingly began integrating lab use with classroom instruction as they 
became more familiar with the system and the software, effective use was not 
maximized. In addition, th'^ system itself did not perform as the teachers 
expected in terms of its promise of individualized instruction. The review of 
student histories showed that students went through identical lesson patterns 
but at different times; little branching to remedial lessons was evident. 

Software Review : The ESC software meets many criteria of gocd design 
both in curriculum presentation and programii.^ng style; however, improvement in 
some areas, especially in the counting of second and third tries as correct 
responses, is needed. 

Observations : The ESC system is reliable and easy to use. Students 
like going to the lab and use their scheduled time effectively. Not all 
teachers, however, used their time in the lab optimally. Many looked at the 
lab as an opportunit:y to do other tasks. 

Teacher Intervlfw and Questionnaire : Although teachers we: very 
positive about their experiences with the ESC lab, the common frustration of 
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an overloaded school day was given as a drawback to having uhe lab. Few 
indications were given that teachers had moved beyond the mechanical use of 
the lab as a separate activity to integrating its use with the rest of the 
school day. Both of these factors would inhibit effeccive use. 
Responses to questionnaire items were varied, indicative of the range in use 
by the teachers. None of the teachers had tried more than half of the 
programs themselves. Logically, it would be difficult to plan for integration 
or assist students if one had litcle familiarity with the programs. All 
teachers did, however, feel tha. ESC was a good supplement to their classroom 
instruction and a positive experience for their students. Most did not want 
the lab removed. 

Student Questionnaire: Students recognized that they could skip reading 
screens or guess and the computer would still give them positive messages. 
Students expressed frustrations when i:hey had to repeat programs (a result of 
being cut off in the middle of a program because of a time factor rather than 
because of poor performance). In spite of these reactions, students wre very 
positive about their experiences and felt working in the lab helped them 
learn. 

BSI Test: Although most students showed growth from October to May 
based on the ESC level placement, results of the comparison of the students' 
working levels at the final testing compared to the testing level were 
problematic. Students were working anywhere from lower than to three or more 
levels higher than their final BSI level score. Such results raise serious 
questions about the use of the BSI as an accurate measure of a student's 
ability. 

CTBS- Although preliminary comparisons of pre- and post- test resuj.ts 
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showed that all but one group showed greater gains than expected (froiu 4 to 20 
scale score points), there were no significant differences by treatment. A 
significant difference was found in mathematics for race, with whites 
performing better than minorities in all categories. Analyses of the re:»ding 
component of the CTBS show mixed results. Students, especially those in the 
lowest quartile, made overall improvement. Results, however, were not 
significant between the treatment groups except for whites in reading 
ccmprehens ion . 

Analyses of these test results were conducted using scaled scores as the 
basic unit since they allow for comparisons across grade levels. Figures 1 
and 2 depict the pre/post- test results for each treatment. 

[ INSERT FIGURES 1 AND 2 HERE ) 

Table 1 shows the difference between actual scaled score gains and 
expected gains based on the pre- test. The expected gains were calculated by 
usinp the pre-test standard scale score to determine percentile rank based on 
fall „orras. Based on the equipercentile assumption, students are expected to 
remain in the same relative ranking frora fall to spring; and, thus, this 
percentile was used to determine the t/^pected" standard scale score from the 
spring norm tables (13. S. Department of Health, Education, and Welfare, 1975). 




Pre/Post Test Results for Mathematics 

Grades 4, 5, and 6 




Pre/Post Test Results for Reading 

Grades 4, 5, and 6 
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Table 1 



Comparison of Difference Between Expected and Actual 
Post-Test Scaled Scores for Mathematics and Reading Tests 
by Grade Lavel and Treatment 



Gsrida/Treatmenr 


Mathematics 


Reading 


A Mathematics 


+8 


-lA 


A Reading 


+6 


-*-6 


5 Mathematics 


+15 




S Reading 


+10 


+15 


6 Mathematics 


+6 


+1A 


6 Reading 


+7 


+11 



According to these results, all groups except the fourth grade group which had 
the mathematics treatment scored higher than expected on both subtests of the 
CTBS based on their pre -test scores. The fourth grade mathematics group 
scored lower than expected on the reading subtest. Th.ere were no significant 
differences between control and experimental groups, except for fourth grade 
reading. 

DISCUSSION OF RESULTS 
VJHAT IS THE HATCH BETWEEN THE FOLLOWING CHARACTERISTICS OF THE ESC EDUCATIONAL 
SYSTEM AND CURRENT LEARNING THEORY? 

Sound Cu rriculum : The ESC curricula were designed according to the most 
current educational thinking. The mathematics curriculum is based on the 
recommendations of the National Council of Teachers of Mathematics (1980) and 
the reading curriculujn on the recommendations of the National Assessment of 
Educationa) Progress (1985). By addressing the recommendations of these two 
groups as the curricula were developed, ZZk. laid the f andation for a sound, 
meaningful program. The developers of the lessons wc^e established 

9 



ERLC 



12 



professionals in the field who organized the units into the discrete, 
hierarchical objectives promoted as basic to sound curriculum development. 

As reported by the National Center for Educational Computing (July, 
1988) , "software can enhance tne educational process most if it is being 
synchronized with curriculum that is being used in the school district." 
Because of the high match with this particular school system's objectives 
(89X), and, more importantly, because teachers recognized the match and were, 
thus, more inclined to view the lab as beneficial, the eftect on learning was 
likely positive. 

Motivation: Students like CSC and the experiences they have had with 
the system. They find it fun y.-^t educational, challenging but not 
frustrating. They wish they could go to the lab more often and st^,y longer, 
and they definitely want the lab to remain at Denbar. Teachers' observations 
and coT^ments confirmed the students' feelings, and they named motivation as 
one of the benefits of using the system. This increased motivation may be tne 
result of the organization and presentation of the ESC lessons, may be caused 
by the use of the computers themselves, or may be because the rest of the 
school day is exce. /ely routine and unstimulating. Although the source may 
be unattributable , ti.e increase in motivation was present and most likely had 
a direct or an indirect effect on the students and their achievement by 
increasing their interest, concentration, and opportunities for learning. 

Active Learning: Observations in the iab showed that students were 
actively engaged in interacting with the computer programs. This engagement, 
however, may have been superficial s^'nce students admitted to not always 
reading the screens, not usin^ the review feature, and often juit guessing so 
they could go on. Although the ESC syscem does provide some aspects of 
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learner control (the review feature, speed changes, and opportunities to 
choose whether or not to repeat an activity), the students need to be taught 
how to ure these features effectively and then monitored as to this use. Such 
instruction and followup monitoring would provide the instructional support 
necessary to guide students to "mindful" control of their own learning 
(Shavelson & Salomon, 1985; Kinzie, 1988). 

Mult i -media Approach: Another very positive factor in providing 
students with increased learning opportunities, is the variety of activities 
and presentations through which the ESC objectives are preseiited. Animation, 
voice, graphics, and other features of ESC are appealing to the students and 
help them focus on learning. Teachers generally felt these features 
contributed to increased learning and named them as a benefit of the system. 
Students generally liked the special features on the ESC system. Although the 
multi-media approach seems beneficial because the variety should reduce rote 
learning and. thus, boredom, this factor could have posed difficulty for those 
students who cannot move easily among multiple methodologies. Some students 
could have focused more on what they were "doing," rather than what the;- were 
"learning. " 

Individualized Tnstruction /Manaeemenf Student histories indicate that 
the ESC programs contain little branching and individualization in terms of a 
program adapted to the needs of the student. Almost all students go through 
an identical sequence no matter what their response patterns. In addition, 
teachers made few changes in the programs of their students. The ESC programs 
do not have sufficient branching to provide the individualization needed to 
meet all a student's needs. Although the sequence of instruction is similar 
for all students, at least it is adapted to the level and pacing needs of each 
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student. In addition, because the computer controls most assignments, the 
constraints on teachers of time and orgar izational management are eliminated, 

I 

liabilities often mentioned as a hindrance to effective use of computers 
(Cosden et al.. 1987; Semmel, 1987). 

Feedback: The review of the ESC software programs showed that a variety 
of feedback procedures were employed: simple yes/no, yes with a reinforcement 
of why correct or some additional information, no with an indication of the 
error, and a no with a branching to a mini-lesson. S<-ndent5 generally felt 
that the computer gave them helpful explanations when they were wrong. In 
most instances this feedback appeared to be presented in a manner that would 
be effective. However, students often were given a false sense of their 
success because of the way correct responses were scored. Students egreel 
that the computer often said "Good Job" when not deserved, and teachers 
mentioned this factor as a limitation to the system. This problem of 
"inflated reporting" occurs because students are able to guess, often without 
a noticeable penalty. 

Time on Task: Whenever, this researcher was in the lab, she saw 
students enter the lab quickly and almost immediately become engrossed in what 
they were doing. Thus, the engagement criterium in Fisher's definition of 
Academic Learning Tine (Fisher et al.. 1980) is met. However, the second part 
of the definition requires that the student be engaged at a level that will 
yield a high success rate. The results of the BSI analyses, showed that many 
students may not have been working at their instructional level. This, 
coupled with the problems H ^scribed in the section on feedbac!". indicates that 
the records of individual students ^houla be carther analyzed to determine the 
effect of "misplacement" on testing resulcs. 
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HOW WAS THE ESC SYSTEM IHPLEMENTED IN THIS PARTICULAR ELEMENTARY SETTING? 

Overall the ESC system and the Tandy network of computers were reliable. 
Down)ime was minim-^i When the lab operator did experience some difficulty 
witb sever.*! of the ESC programs, she was able to get immediate assistance 
through the telephone ir the lab. Besides tlii reliability of the equipnient, 
another. . ^ason few problems existed is students are kepn so busy in the lab 
that they do not have the time nor the inclination to "fc^l around" with the 
equipment. Such behavior is alco hampered by the dedicated presence the 
lab operator and teacher. 

Generally , t eacher.s were positive about both the training and ongoing 
support provided by ESC and their school system. Structured time to look at 
and discuss the programs and how to interpret and use the info.Tnation on the 
various reports is needed, however, if teachers are to maximize the 
effectiveness of the system. Teachers rarely sat down and went through the 
various programs chemselves. Without this background information, they cannot 
easily integrate the ESC lessons with those in the classrooms; they cannot 
anticipate difficulties students might have and, thus, better prepare the 
students for those particular programs; and they cannot offer productive 
ongoing assistance. In spite of good intentions, teachers did not find the 
time to sit down and preview all the programs. 

Most of the teachers did not feel going to the lab was disruptive to 
their instructional program. Although mention was made about fragmentation, 
ESC was looked upon as ^ "worthwhile intrusion." Teachers seemed willing to 
work out their concerns rather than suggesting that the program is too 
intrusive. Since teachers art the prime implementors of any educational 
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Innovation (McUiUghlin, Berman, and liarsh, 1978), such reactions to computer 
use can be viewed as positive. 

The teachers admitted to few changes in their classroom instructional 
program because of the use of ESC. Part of this may be lack of experience 
with the system and its programs. Another reason may be that they were unsure 
of whet kind of changes and how many they were allowed to make because of the 
resear n project. A third reason might be that, as with the introduction of 
any nrw ^ ,o£;ram, participants have certain stages they must go through before 
full integration is possible (Loucks & Hall, 1979). In the interviews, 
howe\er, there were indications that teachers were thinking of ways to use ESC 
more effectively next yea^. 

Several teachers commented in the interviews that their students look at 
the lab as a separate activity. This may be a direct reflection of the 
minimal crcrdination that the t^. ?cher does between classroom and the lab or 
tne lack of interest many teachers demonstrated by their behavior in the lab. 
As pclated out (Cosden et al., 1985), teachers need to maintain an active 
pre:. r.ce since they, not the software, make the distinction between students 
who are working and those who are not. Students ne^id to see the use of the 
lab as an integral part of their classroom learning. Students need to be 
monitored as they are using the lab to minimize guessing, to demonstrate 
teacher interest, and to enable teachers to give ready assistance. In 
addition, teachers need to discuss the programs and the reports with the 
students, thereby giving thew essential feedback on their performance. 
WHAT IS THE EFFECT ON THE ACHIEVEMENT OF STUDENTS USING THE SYSTEM FOR 
MATHEMATICS OR READING INSTRUCTION? 
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BSI 

Concern was expressed by the teachers that the BSI is not a reliable 
placement test. If the BSI does not yield an appropriate placement for the 
students, educational opportunir.ies are not maximized. If students are placed 
too low, they run the risk of being borei and becoming careless in their 
responses. This is in addition to not being able to increase their learning. 
If students are placed too high, they may have skipped skills or concepts 
basic to their current lessons and may become -frustrated. In either case, 
students are not working at their instructional level and cannot make expected 
progress . 
CTBS 

Two analyses of the results of the CTBS showed promise. First, all but 
one group did better than expected on the post- test, regardless of treatment. 
Reasons for a lack of difference are discussed later; however, tne improvement 
demonstrated by most groups is vorth noting. The one group that did not 
perform as expected was the fourth grade group who had the mathematics 
treatment. That group scored fourteen points lower than expected in reading, 
while the reading experimental group scored six points higher than expected. 
No reason for such a discrepancy could be determined; however, it can be 
hypothesized that the difference is the result of the ESC reading curriculum. 

Second, the results of the quartile distribution analyses conducted on 
the scores of only the students who had each treatment showed promise for 
particular groups of students. Students in the bottom quartiles in both 
treatment groups, but especially for reading, made substantial gains from pre- 
to post-tests (g^-'iater than might be expected, including the effects of 
regression to the mean). Often teachers do not work with students ac their 
ins true tiona"* level, especially if Aiej are far below grade level. The fact 
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that students were able to work at their own level in the lab (up to four 
levels below their grade level) may have contributed to these changes. 

Several factors related to the design of the ESC programs could have 
contributed to few statistically significant results. These include the BSI, 
the feedback structure, and ESC's reliance on teacher management. 

As explained in the previous section, the BSI may not provide an 
accurate measure of students' abilities. If students are misplaced in the 
sequence of programs, progress may be hindered. The test results of the 
experimental group would reflect this misplacement. Even if placed at the 
correct level, students may not make maximum progress because, based on the 
way the system is structured, they automatically are assigned all units. 
Instructional time is lost as students complete a unit containing objectives 
already mastered. 

In addition, as students are working through the various programs, they 
may be given a false sense of their achievement when they are "rewarded" for 
guessing, and may, thus, be less motivated. This problem of lack of 
appropriate feedback is compounded if teachers do not monitor students' 
results and recycle the students through programs or units in which they did 
poorly. If skills or concepts that were to be mastered in those units are 
needed for subsequent units, the problem increases. 

Finally, a number of students scored at the highest level on the initial 
BSI. These students, therefore, were cycled through lessons that, rather than 
being instructional, may have only provided practice for skills and concepts 
already known. Thus these students' post- test scores could not be expected to 
be markedly different from their pre-scores. 

I 

Implementation factors that could have contributed to the achievement 
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results are: the one year time span, the differences among teachers both in 
the way they integrated lab learnings with classroom work and in their 
classroom teaching, and the lack of monitoring of the students by some 
teachers and of some teachers by the administration. These factors are 
discussed below. 

This was the first year of the program. Students were expected to learn 
both the system and the content during this period. Although these students 
hed worked with computers before, this was their first experience wxth a 
system that required input through the mouse, as well as through various types 
of keybonrding responses. In addition, since the lessons were written by 
different programmers, types of responses varied. Content, in many cases, was 
new to the st de-ts, especially in tha higher ordered thinking skills. 
Students were entering a K-6 program at a high level without the benefit of 
the skill-building that should have occurred through the earlier levels. 

Teachers' inexperience could have had an impact also. It was not until 
mid -year that it was determined that the computer system was not designed to 
make decisions regarding students' programs, i.e., to not move them forward if 
they had difficulty with a particular unit. Since teachers were not familiar 
with all the programs, they were unable ;o anticipate problems and prepare 
their studenus for these difficulties or to effectively integrate ESC with 
their classroom teaching. 

The second area under implementation relates to individual differences 
among teachers. Teach.-^.rs are individuals whose teaching reflects their 
individuality. Some liiay have enjoyed going to the lab more than others or may 
have taken th^, responsibility of the lab more seriously. Thus, special 
efforts could have been expended in the subject areas of their lab assignment. 
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Differences also can be found in classroom teaching. Teachers may have put 
more effort into the control group subject ar^a because they felt they needed 
to compensate for what they perceived as the extra attention given to tu^ lab 
subject area. In either case, the results of the study may hav? ^ 
compromised. 

The third area under implementation that could have affected the results 
is the lack of or inconsistetit monitoring. As an innovation is being tried it 
is important that the technique or program be used as intended by the company 
and that such use not alter over the course of the study (Marshall, 1989). As 
is obvious from the discussion so far, the ESC program was not implemented 
according to all company specifications. These discrepancies occurred through 
misunderstandings, as well as the inevitable drifting of time and attention 
that can occur over time if implementation is not monitored periodically. 
Thus, sTome teachers were not reading and acting on the various reports, were 
not following prescribed procedures in the lab, aid were not encouraged to 
maximize the integration between the laL and the c^ ^ssroom. 

The third explanation for few statistically significant results is the 
inter-relatedness of the content of the two curricula. Although ail classes 
(with the exception of the one fifth grade class) used either the reading or 
mathematics curriculum, both emphasize thinking skills and develop good te<:t 
taking skills by stressing careful reading and attention to detail. In 
-addition, reading skills are promoted through the mathematics curriculum, and 
several of the reading units contain machematics concepts. Thus, it is 
possible that the learning that occurred as a result of the one curriculum the 
students were placed in transferred into the test results of the other 
curriculm. 

1^^ 



Although precise reasons cannot be given for a lack of statistically 
significant differences between the results of the control and experimental 
groups, the above descriptions of interfering factors offer a viable 
explanation. Some cau be controlled; others may be inevitable in a real-world 
setting. Because of this, practical significance or other issues may be more 
important and useful measures of the success of the program. 

SIGNIFICANCE OF RESULTS 

1. Because this study provides a more realistic assessment of the 
effects of ESC on the practices at this site and the effect of the system on 
the students and teachers, decision-makers interested in research on such a 
system can read this study assured that the results may more readily transfer 
to their own systems. 

2. ESC Incorporates in its design and implementation many concepts or 
idea.s that learning theory says are important in education. ESC provides a 
structure through which individualized instruction is possible. As classes 
get -ore diverse, learning systems such as ESC will enable teachers to more 
easily meet the needs of these students. Students can be kept attentive and 

task while the teacher works on a more personal basis with other students. 
T.iey will do so because the programs are attractive, attention-focusing, and 
motivating. 

3. Becau e ESC correlates their objectives with those of the school 
system, teachers are able to see the relations? p between their classroom 
responsibilities and the work in the lab. Teachers should be note inclined to 
integrate lab work with their classroom instruction. 

4. ESC is a relatively easy program to implement. Training is provided 
by the company. An aide manages the lab. Because all the programs are on CD 
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ROM and the system is networked, time is not wasted booting up individual 
programs on diskettes. A management program allows the aide to program an 
entire day so transitions between classes are almost transparent. Special 
teacher requests can be met easily. By removing all "housekeeping" and 
management responsibilities from the teacher, ESC increases the probability of 
acceptance of the lab by the teacher. 

5. Although the comparison of the test results of the control and 
experimental groups did not yield statistically significant results, two other 
analyses may be of practical significance. First, cll but one group did 
better than expected on the post-test, by as many as 20 standard scale points; 
second, students in the bottom quarter grouping had an average gain of over 
ICO standard scale points (approximatly two standard deviations) from the pre- 
to post -test. These two factors may be more important than the tests of 
significance, given the numerous extraneous variables which could have 
affected the results. 

CONCLUSION 

Integrated learning systeL with both CAI and CMI coniponents offer the 
educational advantage of sequential, basic skills curricula complemented by a 
method for monitoring individual students. The success ot these integrated 
learning systems depends on the quality of each component. The evaluation of 
ESC focused on the curricular scope and sequence and its relation to the 
program in Denbar, how the management system was used by the teachers to 
assess student weaknesses and plan appropriate followup, general student and 
teacher reaction to using the system, and educational outcomes. 

Overall, ESC measured up to the expectations in each of these areas. 
The curricula closely matched chose the teachers were using; teachers and 
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studetits responded very favorably to the program; tast scores indicated that 
students were learning; and, although the reports generated by the management 
system were not used as expected, they did contain the necessary information 
to make appropriate instructior^^i. decisions. In addition, the developers of 
ESC have responded very favorably to feedback from the school and are making 
needed adjustments to the system. The use of the ESC learning system has been 
a positive experience at Denbar, but its potential has not yet been fully 
realized. As teachers becomp more familiar witn the system, they should be 
able to integrate its use with their classroom teaching more effectively; and 
as students begin to build on the skills being developed, their achievement 
seres should continue to improve. The program should remain at Denbar, but 
Lts Implementation needs to be monitored to ensure that effective use 
continues to be maximized. In the meantime, results of this study are 
sufficientl- ^sitive to encourage other school systems to implement ESC and 
undertake and report on their own studies of its effectiveness, thereby 
Increasing the body of research in the use of integrated learning systems in a 
school environment. 
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